Real-Time Scanning Charged- Particle Microscope Image Composition with 

Correction of Drift 
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In this article, a new scanning electron microscopy (SEM) image composition technique is de- 
scribed, which can significantly reduce drift related image corruptions. Drift-distortion commonly 
causes blur and distortions in the SEM images. Such corruption ordinarily appears when conven- 
tional image-acquisition methods, i.e. "slow scan" and "fast scan", are applied. The damage is 
often very significant; it may render images unusable for metrology applications, especially, where 
sub-nanometer accuracy is required. The described correction technique works with a large number 
of quickly taken frames, which are properly aligned and then composed into a single image. Such 
image contains much less noise than the individual frames, whilst the blur and deformation is min- 
imized. This technique also provides useful information about changes of the sample position in 
time, which may be applied to investigate the drift properties of the instrument without a need of 
additional equipment. 



INTRODUCTION 

Advances in fundamental nano-science, development 
of nano-materials, and eventually manufacturing of 
nanometer-scale products all depend to some extent on 
the capability to accurately and reproducibly measure di- 
mensions, properties, and performance characteristics at 
the nano-scale. Scanning electron microscopes (SEMs) 
have been used in this application for many years [H, 
Q. Since progress in nano-science and nano-technology 
has been rather rapid recently, the dimensions of nano- 
structures and nano-objects have shrunk significantly. 
Consequently, accurate SEM imaging has been empha- 
sized. The dimensions or distances have been measured 
from SEM images or line-scans. Current imaging meth- 
ods in SEM are often incapable of achieving the desired 
accuracy, because the SEM images, at such high mag- 
nifications, often suffer from drift-related distortion. In 
many cases, the drift is significant and the SEM images 
exhibit deformations or blur. The same problem is also 
experienced in other fields, e.g. scanning probe micro- 
scopies. 

Several correction methods are being developed that 
compensate for these effects. Some work on correcting 
the time-dependent drift distortions has been performed 
in fields similar to scanning electron microscopy ((j, 
[l3|| . A research in drift-distortion evaluation and 
correction in SEMs has been published in [l2j , , [Toj| . 
Technique described in these papers covers correction 
in images with slow drift and low magnification. The 
overall imaging times are high, reaching tens of min- 
utes. The magnification does not exceed 10000. Tech- 
nique for very fast SEM or Scanning helium-ion beam 
microscopy, where signal-to- noise ratio (SNR) may drop 
below 5 x 10 _1 , is still needed. This manuscript describes 
a possible correction method based on composition of 
drift-distortion corrected SEM images. The technique 
uses cross-correlation for displacement detection. It not 
only provides more accurate images, but also sample po- 
sition information, which can be successfully employed in 



diagnostic applications. The method is implemented as a 
software program in the C programming language. With 
this approach, the solution is fast, multi-platform, multi- 
processor capable, and moreover can be easily integrated 
into the majority of the SEM software. 



DRIFT EFFECT ON IMAGES 

In the SEM, the image is formed by scanning over the 
sample and acquisition of an intensity value at each loca- 
tion on the sample corresponding to a pixel in the image. 
The intensity value £(r) depends on the landing position 
of the electron beam (on the sample) r. Most SEMs use 
the raster pattern for scanning over the sample. Let the 
raster pattern be defined by the time-dependent vector 
function: 
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where t is time, M is a constant of the length on sample 
corresponding to a single-pixel step, x and y are column 
and row indexes in the SEM image. e x and e y are the 
unit vectors in x- and y-direction, to is the dwell time of 
one pixel, td is the dead time between two pixels, tj is the 
time needed to move the beam to the beginning of the 
new line. [q\ is a symbol for the floor (#) function as used 
in programming languages. X and Y are the pixel- width 
and pixel-height of the SEM image. These equation are 
in agreement with those published in [l2j . 

Imaging in the SEM may be defined as a relation be- 
tween the intensity map of the sample £(r) and the SEM 
image I(x, y): 



i(x(t), y (t)) = mm)- 



(4) 





FIG. 1. Illustration of drift-distortion-related image corrup- 
tion on simulated @] "slow scan" SEM images of a gold-on- 
carbon sample, (a) Ideal, undistorted image, (b) Typical 
corrupted image. 

The relation between / and £ may in practice be very 
general. For simplicity, let K be a constant in this pa- 
per, since this does not affect generality of the described 
technique. In the ideal case r(t) = r r (t); however, drift 
and space distortions are always present in scanning mi- 
croscopes and they often significantly affect the position 
r: 

f(t) = r r (t) + D d (t) + D s (f r ). (5) 

The space distortion D s is constant in time and may 
be simply compensated for, when its function is known. 
This kind of distortion is caused by non-linearities in de- 
flection amplifiers and appears mostly at low magnifi- 
cations. On the other hand, the drift distortion is 
changing in time, its function is usually unknown, and it 
most significantly affects the high-magnification images. 
The drift distortion may arise from several sources; e.g. 
translational motion of the sample, tilt or deformation of 
the electron-optical column, outer forces and vibrations, 
or temperature expansion. High-magnification images 
are very sensitive to drift distortion, since microscopic 




FIG. 2. Illustration of the composition (averaging) of dis- 
placed image frames, (a) Composition of a few image frames, 
(b) composition of a large number of frames, the image ex- 
hibits excessive blur. 

displacements, tilts, or temperature changes can easily 
cause nanometer distortions and displacements, which 
can significantly impair the SEM image and its usabil- 
ity for nanometer-scale measurements. 

The drift-distortion function is generally unknown, 
however, since it characterizes motion of physical bod- 
ies, it must be continuous and thus square-integrable. 
Therefore, drift-distortion function may be expanded to 
Fourier series: 

oo 

D cd (t)= c « e ~ in '> (6) 
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D d = $t(D cd )e x + S(D cd )e y , (7) 
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where c n are the (complex) Fourier coefficients, U is the 
overall energy of the drifting system. Since U is lim- 
ited, for high n the coefficients c n must be nearing zero. 
In practice, c n for frequencies higher than 200 Hz corre- 
spond to noise only and are negligible. This approximate 
number is based on experimental values. Therefore, the 
D c d(t) can be written: 

N 
n=—N 
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FIG. 3. Illustration of the cross-correlation displacement de- 
tection with noise filtering, (a) Original frame, (b) noise- 
reduced frame, (c) unprocessed cross-correlation function, 



where N represents the highest significant angular fre- 
quency. 



SEM IMAGING METHODS 

In the SEM, the acquired intensity signal always con- 
tains noise. The intensity function is thus a superposition 
of real signal and noise: 



e(r,t) = ^(r)+enW, 



(10) 



where £ s is the position-dependent real signal and £ n is 
the time-dependent noise. This noise is a superposition 
of Poisson noise, originating from the electron source and 
the secondary emission, noise originating from the ampli- 
fier and electronics, quantization-error noise, etc. Due to 
the central limit theorem, it is legitimate to suppose that 
the mean value of this noise is zero: 



>=0. 



(11) 



In order to obtain a SEM image with a desired level of 
noise, the overall pixel dwell-time to must be sufficiently 
high. Unfortunately, the electron yield is usually low and 
the to must often be set to times ranging from tens to 
several hundreds of /is. 

There are two common techniques to achieve this in the 
SEM, i.e "slow-scan" and "fast scan". With the "slow- 
scan" method, the image is acquired within a single scan. 
The acquired value is in this case: 

to+t]j 

I(*(t ),l/(*o)) = ^ J (6W))+Cn(*))t, (12) 
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(13) 



r(t) = r r (t ) + D a (r r (t )) + D d (t) = 

= const + D d (t). (14) 

The noise is reduced by long integration as shown in 
Eq. ([T3]) . Required level of noise determines the dwell- 
time to- Since the desired beam position does not change 
during the acquisition of a single pixel, the only changing 
component of the position (Eq. J5J)) is the D d as stated 
in Eq. (jHJ). In practice, the drift-distortion-related dis- 
placements are not significant between two pixels, be- 
cause the time t p is still not long enough. However the 
line- acquisition time Xt p +tj may already be much larger 
than the period of the highest frequencies. Therefore, if 
the "slow-scan" technique is employed, the line-scans and 
thus also the images may be significantly distorted (See 
Fig. [1]). The distortion is time-dependent, and thus dif- 
ferent for each line and image and cannot be corrected, 
unless additional information about the drift-distortion 
function Dd(t) is provided. 




FIG. 4. Demonstration of the method on real SEM images of the gold-on-carbon resolution sample. Horizontal fleld-of-view is 
441 nm for all images, (a) single acquired image with the pixel dwell time 50 ns, (b) composition of 10 images, (c) composition 
of 20 images, (d) composition of 40 images, (e) corrected composition of 120 images, and (f) Plain average of the same 120 
images. 

The other common imaging method in SEM is the tion during this time is negligible and the integral (fT2j) 
"fast-scan". The image is composed of multiple (A^) can be approximated as constant. The image pixel value 
frames, for which averaging is the mostly applied tech- 
nique. The frames are acquired with the lowest possible 
pixel-dwell time to- Since, in practice, the to can be set 
to as low as 25 ns, the change in the drift-distortion func- 
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FIG. 5. Illustration of the image composition procedure. Boxes denote entities like images or numbers, arrows indicate 
processes. The letters (a) — (m) represent individual steps in the procedure. 



is then an average of corresponding frame-pixel values: 

I k (x(t ),y(t )) = KUr(t + kt f )) + 

+ K£ n (to + kt f ), (15) 

1 Ni 

I(x,y) = —J2l k (x,y). (16) 

1 k=0 

t f =Y(Xt p + t j )+t jj , (17) 

tf is a time period between beginnings of acquisition of 
two following frames, tjj is the dead time between the end 
of acquisition of one frame and beginning of the next one. 



Considering Eq. (fTTj) , the higher N{ , the lower noise level 
is present in the composed image. The required noise- 
level thus determines the number of composed frames 
Ni. For high Nf. 

Ni-l 

tn(to + kt f )^0. (18) 

k=0 

Because the scanning raster pattern is constant for all 
frames, 

r r (t + kt f )=f(to). (19) 
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Eq. (fT5j) may be expanded: 

K Ni_1 

I(x(t ),y(t )) = — Urr(to) + 

1 k=0 

+ D s (f r (t ))+D d (t + kt f )}. (20) 

With current SEMs, the frame-acquisition time tf can 
be much lower than the period of even the highest drift- 
distortion frequencies. The drift-distortion within the 
single-frame acquisition time is then minimal. However, 
it becomes significant during acquisition of the whole im- 
age, especially, when the dead times tjj are high, which 
is the case even with many current instruments. Con- 
sidering drift effects negligible within a single frame, the 
drift affects all image pixels equally. Point-spread func- 
tion (PSF) may be constructed from the function Dd(t). 
Such a PSF consists of multiple separate points, which 
produces blurry images similar to Fig. [2j The PSF can 
not be used for deconvolution-based drift-distortion cor- 
rection, since it is unknown like the Dd itself. 

INTER-FRAME DRIFT-DISTORTION 
CORRECTION 

The "fast-scan" method may be significantly improved 
using drift-distortion correction. This is possible, when 
the frames are taken during short enough times and 
Y(Xt p + tj) <C 2tt/N. Since the space-distortion D s is 
much less pronounced and much smaller that the drift- 
distortion Dd at very high magnifications, it will be ne- 
glected from now on. The Eq. ([20]) then becomes: 

K iVi_1 

I(x,y) = — Ur r + D dk ], (21) 

1 k=0 

D dk = D d (t + kt f ). (22) 

The image is in this case the mean value of Ni displaced 
images. 

Fortunately, under certain conditions, it is possible 
to find the displacement vectors of the images, which 
are equal to the drift-distortion values D d k- These vec- 
tors then can be compensated for and thus the drift- 
distortion can be corrected. One possible approach is a 
cross-correlation-based displacement detection, which is 
used in this work. However, choice of the method deter- 
mines the requirements, which may include low-enough 
noise, well-pronounced image features, etc. The complete 
set of requirements will be addressed in future publica- 
tions. 



CROSS-CORRELATION WITH NOISE 
REDUCTION 

If two image frames / and g contain similar fea- 
tures at different positions, the cross-correlation inte- 
gral has a large value at the vector corresponding to 



the displacement of the features. The SEM digital im- 
age frames are in this application represented by dis- 
crete two-dimensional real functions. Therefore, the two- 
dimensional discrete cross-correlation is applied. 

If the image frames are noisy, the peak in the cross- 
correlation function becomes overridden by numerous 
other peaks, corresponding to random correlation of noise 
(Fig. [3k) This often makes finding the displacement vec- 
tor impossible. This could be tackled by low-pass fre- 
quency filtering. This can be performed in the fre- 
quency domain. The cut-off frequency is determined 
by the filter-radius R. Although, this filter significantly 
wipes out all high-frequency features from the image, 
and therefore it is inapplicable for general reduction of 
noise, it still works very well for the total-maximum 
search of a two-dimensional function. The maximum of 
the cross-correlation function becomes higher above the 
background and it is easy to find it. (See Fig. E31). 

According to the cross-correlation theorem [7j, the 
cross-correlation can be calculated using the Fourier 
transform. The widely used FFTW3 [3| algorithm is ap- 
plied for Fourier transform calculations. In order to speed 
up the calculation, the cross-correlation is combined with 
the frequency filtering. (See Fig. [5^.) The conjugation 
and multiplication is done only in the central circle (the 
lowest frequencies) of the Fourier image, while the rest is 
zeroed: 

J = { [F{fif))Y ■ F(g(r)) if M < R, , 23) 

1 otherwise. ^ ' 

7 = F- 1 (J) (24) 

Then, the inverse Fourier transform is applied and the 
noise-reduced cross-correlation image is obtained. For 
every pair of frames, only two forward and one inverse 
Fourier transforms are needed, while one of them is also 
part of the next pair. 

SUB-PIXEL-ACCURACY DISPLACEMENT 
DETECTION 

In order to find the displacement vectors, the maxi- 
mum of the cross-correlation is searched for. Since this 
method is sample-dependent, there may appear prob- 
lems finding the displacement vector corresponding to 
the transition. For example, if the image contains pe- 
riodic features, there are several maxima in the cross- 
correlation image. In case of large blur or noise, the peak 
may be very wide and the uncertainty in the position of 
the maximum may be very high. 

Plain search for maximum provides just single-pixel 
accuracy of the displacement- vector. The peak may 
be interpolated by a suitable function, which enables 
for calculating the displacement vectors with sub-pixel 
accuracy. The third-order two-dimensional polynomial 
function with coefficients fco — ^9 was chosen as a suit- 
able function for the interpolation. The coefficients may 
be found by polynomial fitting using the common least 
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FIG. 6. The results of the corrected composition compared 
with the uncorrected "slow scan" demonstrated on a real SEM 
image of a gold-on-carbon resolution sample. Horizontal field 
of view is 422 nm. (a) Slow-scan image with the pixel dwell 
time 300 fis. (b) The result with the new technique, (c) Dif- 
ference between the two images. 



squares method, which is widely used in similar appli- 
cations. The Cholesky factorization [4J is applied to 
speed the calculation. The maximum is then numerically 
found; its position is the searched displacement-vector 
with sub-pixel accuracy. However, the exact accuracy- 



value calculation is not covered in this article. 

Since the displacement- vectors are calculated with sub- 
pixel accuracy, it is reasonable to shift the images with 
sub-pixel accuracy as well. Let the displacement-vector 
be s 

— Sx^-x H~ s y e y The shift can be performed with 
sub-pixel accuracy using the Fourier Transform, because 

F s (u, (j)) = F(cj, </>) exp(-iux s - i<jyy a ). (25) 

The obtained displacement- vectors with the sub-pixel 
accuracy obtained in the previous step are used for 
shifting, which compensates for the inter-frame drift- 
distortion. The corrected images are then pixel-by-pixel 
averaged together, however, other composition methods, 
e.g. median filtering, can be also used. 



The procedure of the method is illustrated in Fig. [5] 
The process starts with acquisition of two frames; A 
and B (Figs. [5^ and b). In order to minimize the influ- 
ence drift-distortion on the frames, these must be taken 
with minimum pixel-dwell time possible, which is usu- 
ally limited by the instrument. Both frames are then 
converted into the frequency domain (Figs. [5h and d). 
These frequency-domain images are then conjugated and 
multiplied, which is combined with frequency filtering 
(Fig. Et). This results in a cross-correlation image in the 
frequency domain. Then, the cross-correlation image in 
space domain is obtained (Fig. 030 . The cross-correlation 
image is interpolated by the third-order two-dimensional 
polynomial function. This enables finding the displace- 
ment vectors with sub-pixel accuracy (Fig. [5^). The 
coordinates of the maximum denote the found displace- 
ment vector (Fig. Eh). This displacement vector is used 
to shift the image B in its frequency-domain represen- 
tation, which enables the sub-pixel accuracy alignment 
(Fig. [5}). Shifted image B is converted into the space do- 
main (Fig. El) and averaged with the image A (Fig. [5k). 
Image A has (except in the first iteration) higher infor- 
mation weight, as it already represents a sum of multiple 
image frames. If the SNR is not sufficient (Fig. 03), the 
composed image is copied into the frame A (Fig. 03xl) 
and a new frame B is acquired. The process then repeats 
until the SNR is sufficient, or the software runs out of 
frames. 



RESULTS 

The discussed image-composition method was tested 
on gold-on-carbon resolution images (Fig. [4]) and on ar- 
tificial images. A Mac Pro computer with two dual-core 
Intel Xeon Central Processor Units (CPUs) and 4 GB of 
Random Access Memory (RAM) was employed for the 
calculations. The 64-bit edition of Gentoo Linux Oper- 
ating System (OS) was installed on the computer. 




COMPOSITION PROCEDURE 
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FIG. 7. Example of a drift tracking. Subsequent displacement vectors are displayed in the graph, (a) Drift-distortion of a 
sample placed on a fixed stage. Frames were taken every 60 s. (b) Drift distortion of the sample used in Fig. [H Frames were 
taken every 1 s. 



For the real images, the pixel dwell time was set to 
the lowest instrument setting (50 ns). The frame-rate 
was 1 frame per second, which was also the fastest set- 
ting. Single acquired frame (Fig. |4^i) was very noisy; 
only the most prominent features (about 200 nm in di- 
ameter) were clearly visible. A composition of 10 frames 
(Fig. Ht>) already contained visible features in the back- 
ground (about 20 nm in diameter); some inner structure 
of the grains (sized about 5-10 nm) became observable. 
Compositions of 20 (Fig. Eh) and 40 (Fig. SJi) frames 
embodied some more detail. The inner structure of the 
grains, as well as all the background features are clearly 
visible. The composition of 120 frames from the new 
composition method (Fig. |4^) and the existing averaging 
method (Fig. 0f) are included for a comparison of the 
new and the traditional averaging methods. The tradi- 
tionally averaged image was significantly more blurred 
than the image composed using the described method. 
Both images have similar SNR. The final image (Fig. |4^) 
exhibits low noise and high detail whilst preserving the 
shapes and dimensions. 

The slow-scan image is displayed in Fig. [6^. The pixel 
dwell time was 300 /is, which is a common choice for such 
imaging. The image looked clean and noise free on in- 
spection; however, the difference between the slow-scan 
image and the image obtained with the new composi- 
tion method, as shown in Fig. [6t exhibited a significant 
difference, which is believed to be associated with the dis- 
tortions in the slow-scan image. The image was acquired 
for 6000 times longer time than the image in Fig. [4^ and 
the drift-distortion affected the shapes significantly. 



The obtained sequences of displacement-vectors were 
also used to track the sample position with respect to 
the beam (Fig. [7)). This information was very usable for 
drift investigation. In the case of the sample of the fixed 
stage, displayed in Fig. [7^, there was a roughly 70-nm- 
long straight start-up drift followed by a periodical cir- 
cular drift, which was caused by periodical temperature 
changes inside the electron-optical column. 

On the other hand, the Fig.03 shows the displacement- 
vector sequence associated with a typical drift in the 
SEM. Assuming the obtained curve to be associated with 
a relative trajectory of physical bodies with a position 
noise superimposed to it, it is possible to estimate the 
accuracy of the displacement- vector searching to be ap- 
proximately 0.5 nm, which corresponds to 0.5 pixels. 

Speed of calculation is another important aspect of this 
method and its implementation. It was not possible to 
try the technique in a real-time imaging application, since 
this would require integration of the technique into the 
SEM software, which was not possible. However, the 
calculation times were measured and on 512x512-pixel- 
large images, a single search for a displacement- vector 
took in average 0.08 s, while the times of individual frame 
compositions are very consistent. 



CONCLUSION 

The technique is implemented as a computer program 
written in C language, which is the advantage due to its 
optimization possibilities and ease of possible incorpora- 
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tion into SEM software. On reasonably fast computers, 
this program is capable of real-time processing. The algo- 
rithm is well distributable, thus, it is suitable for running 
on computer clusters or multi-core or multi-processor en- 
vironments, including graphics processing units (GPUs). 
The method has been verified on real and artificial SEM 
images demonstrating its usability for true-shape imag- 
ing and for drift investigation applications. It was also 
tested for the calculation speed, which is high enough 



for real-time processing, when integrated into the SEM 
software. 

Since the power of this method strongly depends on 
many factors, e.g. sample-feature shapes, noise, image 
size, etc., its limits should be throughly examined. Cal- 
culation of accuracy and confidence intervals, influence of 
sample charging and contamination are still under inves- 
tigation. These issues will be addressed in future works 
on this project. 
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